Method and computing system for creating and displaying 
images with animated microstructures 

BACKGROUND OF THE INVENTION 

The present invention relates generally to the field of publicity on the World Wide Web and 
more particularly to the generation and display of images with animated microstructures. 

The World Wide Web is becoming an important platform for electronic commerce. Corpora- 
tions and individuals access the Web in order to buy various goods, both immaterial and mate- 
rial goods. In order to attract clients to given Web sites, there is a need for both information and 
publicity. Presently, publicity is achieved by putting banners on Web pages of interest to poten- 
tial clients. Sometimes, in order to attract the clients attention, text banners are successively 
displayed and erased. These "text blinking" actions are distracting the client and annoying him. 

In the present invention, we disclose a new method for delivering publicity and information. 
According to this new method, an image (the global image) incorporates a microstructure 
which may be a text, a logo, an ornament, a symbol or any other microstructure. The micro- 
structure may move, change or evolve over time. The image is either static or may also evolve 
over time. When seen from a certain distance, mainly the image is visible. When seen from 
nearby, mainly the microstructure is visible. At intermediate distances, both the microstructure 
and the global image are visible. Thanks to its inherent artistic beauty and to the way it for- 
wards a message, the new method is attractive to clients. 

Several attempts have already been made in the prior art to generate images incorporating 
information at the microstructure level where from far away mainly the global image is visible 
and from nearby mainly the microstructure is visible. A method hereinafter called "Artistic 
Screening" was disclosed in US patent No. 6,198,545 (inventors : V. Ostromoukhov, R.D. Her- 
sch, filed March 27, 1995) and in the article by V. Ostromoukhov, R.D.Hersch, "Artistic 
Screening", Siggraph95, Proc. Computer Graphics, Annual Conference Series pp. 219-228. 
Another method hereinafter called "Multicolor Dithering method" was disclosed in US patent 
application 09/477,544 (inventors : V. Ostromoukhov, R.D. Hersch, filed Jan. 4, 2000) and in 
the article by V. Ostromoukhov, R.D. Hersch, "Multi-Color and Artistic Dithering", Sig- 
graph'99, Computer Graphics Proceedings, Annual Conference Series, 1999, pp. 425-432. A 
further method for incorporating a microstructure into an image by computing color differ- 
ences is disclosed in European Patent application 99 114 740.6 (inventors R.D.Hersch, N. 
Rudaz, filed July 28, 1999). An additional method for creating microstructures within an 
image relies on a large dither matrix whose successive threshold levels represent the micro- 
structure and uses standard dithering to render the final image (see for example the paper by 
Oleg Veryovka and John Buchanan, Halftoning with Image-Based Dither Screens, Graphics 
Interface Proceedings, 1988-99, Ed. Scott MacKenzie and James Stewart, Morgan Kaufmann 
Publ. or http://www.graphicsinterface.org/proceedings/1999/106/). 

Another approach for embedding a microstructure within a color image relies on the modifica- 
tion of brightness levels while preserving the chromaticity of the image (see US patent 5,530, 
759, Color Correct Digital Watermarking of Images, inventors: W. Braudaway, K.A. Mager- 
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lein and EC. Mintzer). However, in this last method, if the microstructure incorporates large 
uniform surfaces, the global image may be subject to significant changes and the microstruc- 
ture may become visible from a large distance. 

Furthermore, all of these previous methods were focussed on static images, mainly for printing 
purposes and did not disclose how to create attractive dynamic images, i.e. images whose 
microstructure evolves over time. 

The disclosed method distinguishes itself from traditional dynamic text display (for example 
traditional TV spots containing text lines moving across the display) by the fact that in the new 
method, the text is embedded as a microstructure layer within a global image which may be 
totally independent of the text. In addition, when seen from a certain distance, the microstruc- 
ture disappears and only the global image remains visible. 



SUMMARY 

The display of images comprising animated microstructures can be used to create beautiful 
information and publicity sites attracting the attention of clients. Especially for clients visiting 
Web sites, images with animated microstructures are capable of forwarding a message incorpo- 
rated into the animated microstructure. Such images with animated microstructures are espe- 
cially attractive for advertisements on Web pages. The present invention discloses a method for 
creating, respectively, displaying a target image with an animated microstructure, where the 
target image is made of a succession of target image instances which differ from each other by 
an embedded microstructure which evolves over time. The method comprises the steps of 
defining an original image, defining how the embedded microstructure evolves over the suc- 
cession of target image instances, possibly defining a set of basic colors for rendering target 
image instances, possibly defining a warping transformation mapping between a target image 
space containing the target image and the animated microstructure space, and rendering from 
the original image a succession of target image instances comprising the evolving embedded 
microstructure. The warping transformation allows to warp the evolving microstructure in the 
target image. In addition, a mask can be defined which specifies the part of the image to be ren- 
dered with an animated microstructure and the part which is being left unanimated. Further- 
more, a multi-valued mask allows to control the intensity of the microstructure by providing 
the respective weights of the original image and the animated microstructure image in the tar- 
get image. The preferred embodiment for rendering target image instances is a dithering 
method based either on standard dithering or on multicolor dithering. Both dithering methods 
use a dither matrix whose succession of threshold levels specifies shape of the embedded 
microstructure. The selected dithering method may be simply applied to one or several color 
layers of the original image or, if a set of basic colors has been defined, may be applied to one, 
several or all basic colors. The evolution of the embedded microstructure may be defined by an 
animation transformation mapping between an animated dither matrix space and an original 
dither matrix space paved by the dither matrix. The optional warping transformation maps 
positions from the target image space containing the target image to the animated dither matrix 
space. 

In order to render the target image, initialization steps comprise selecting color information 
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necessary for rendering the final evolving microstructure image, selecting a microstructure, 
selecting of a time-dependent animation transformation allowing the microstructure to evolve 
over time and possibly selecting a mask specifying regions of the original image that are to be 
rendered with the selected microstructure. The rendering steps comprise a conversion from 
original image colors to the basic colors selected during the initialization steps and an update 
of the current instance of the animation transformation when a new instance of the target image 
is to be rendered. 

The present invention also comprise an image with an embedded microstructure evolving over 
time, where from far away mainly the image is visible and from nearby mainly the evolving 
microstructure is visible, and where said image is displayed as a succession of image instances, 
each image instance differing from previous image instances by the microstructure evolution. 
The visibility of the microstructure embedded into the image is tuned by a mask whose values 
represent relative weights of an original image without embedded microstructure and a corre- 

^ sponding image synthesized with the embedded microstructure. Mask values evolving over 
time yield apparent changes in the embedded microstructure appearance properties such as the 

^ visibility, location or spatial extension of the embedded microstructure within the image. 

{LJ 1 ; The invention also comprises image animations with an embedded microstructure evolving 
over time, where from far away mainly the image animation is visible and from nearby mainly 
_ " the evolving microstructure is visible, and where, when seen from nearby, frames differentiate 
q themselves from previous frames also due to microstructure evolution. As in the case of an 
%\ image with evolving embedded microstructure, the mask values specifying the part of the 
"H image to be rendered with an embedded microstructure as well as its intensity may themselves 
H evolve over time and for example follow certain moving structures within the image anima- 
p tion. 

r " The invention also comprises a computing system capable of displaying an image with an 
embedded microstructure evolving over time, where from far away mainly the image is visible 
and from nearby mainly the evolving microstructure is visible. This computing system is made 
of a server computing system storing the image as a sequence of image instances and of a cli- 
ent computing system capable of receiving the sequence of image instances from the server 
computing system and capable of displaying this sequence. In another embodiment, the com- 
puting system comprises a server computing system and a client computing and display sys- 
tem, where the client computing and display system receives from the server computing 
system as input data an original color image, microstructure data, microstructure evolution 
parameters and possibly color information and where the client computing and display system 
synthesizes and displays the target image with the embedded microstructure on the fly. 

BRIEF DESCRIPTION OF THE DRAWINGS 

For a better understanding of the present invention, one may refer by way of example to the 
acompanying drawings, in which : 

FIG 1A shows a dither matrix where the microstructure is given by the sequence of dither 
threshold levels, represented in the figure as gray levels; 

FIG IB shows an enlargment of a part of the dither matrix of FIG 1 A demonstrating how the 
dither threshold levels define the microstructure; 
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FIG 2 shows uniform intensity patches dithered with the dither matrix of FIG 1; 
FIG 3 shows an image overlaid with a warping grid; 

FIG 4 shows a mask specifiying the parts of the image to be rendered with animated micro- 
structures (in black); 

FIG 5 shows one instance of the animated microstructure image obtained by multicolor dither- 
ing of the original image shown in FIG 3; 

FIG 6 shows a sequence of 6 other instances of the animated microstructure image; 

FIG 7A shows schematically a comparison between an input intensity signal (or image) P(x) 
and a dither threshold value G(x) and according to that comparison, the setting of a foreground 
or background color; 

FIG 7B shows relative intensities d a , d b , d c , and d d of colors C a , C b , C c , and C d ; 

FIG 7C shows the conversion of relative intensities d a , d b , d c , and d d of colors C a , C b , C c , and 
C d into corresponding surface coverages; 

FIG 8 shows a diagram of elements useful for creating images with animated microstructures; 
FIG 9A shows schematically an original image; 

FIG 9B shows schematically a dither matrix paving an original dither matrix space; 

FIG 10A shows a warping grid laid out in an animated dither matrix space; 

FIG 10B shows the grid of FIG 10A, warped and laid out on top of the target image; 

FIG 11 A shows a mask specifying the part of the target image to be rendered; 

FIG 11B shows one instance of the target image rendered with an animated microstructure; 

FIG 12 shows the warping transform T w (x,y) mapping from target image space to animated 

dither matrix space and the animation transform T t (u,v) mapping from animated dither matrix 

space to the original dither matrix space; 

FIG 13A shows a rectangular grid and the warped rectangular grid specifying the warping 
transform between target image space and animated microstructure space; 

FIG 13B shows a microstructure in an animated microstructure space; 

FIG 13C shows the same microstructure in the target image space, warped by the warping 
transformation defined according to FIG 13 A; 

FIG 14A shows a one-dimensional color RGB image with red, green and blue color intensities 
varying as function of their position on the x-axis; 

FIG 14B shows schematically comparisons between the RGB input intensities of the image of 
FIG 14A and a dither threshold value G(x) and according to these comparisons, the setting of 
the resulting basic colors (red, green and blue); 

FIG 14C shows the colors resulting from the superposition of the basic colors set according to 
the comparison of FIG 14A; 

FIG 15 A shows a one-dimensional color RGB image with red, green and blue color intensities 
varying as function of their position on the x-axis; 

FIG 15B shows schematically the comparison between the blue input intensity of the image of 
FIG 15 A and a dither threshold value G(x) and according to this comparison, the setting of the 
resulting basic blue color; 

FIG 15C shows the resulting target image comprising the original red and green colors and the 
dithered blue color; 

FIG 16A shows a dispersed-dot two-dimensional dither matrix; 
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FIG 16B shows the one-dimensional dithering of constant mask values p(x) with ID dither 
matrix values D(x) and the resulting spatial distribution of microstructure image color values C 
and original image resampled color values C r ; 

FIG 17 shows a server computing system transferring to a client computing and display sys- 
tem an input color image, a dither matrix, an animation transformation, a warping transforma- 
tion, a set of basic colors and a mask layer; 

FIG 18 shows a server system interacting with a designer program or a designer applet running 
on a client computer; 

FIG 19 shows a flow chart describing the steps required to create and obtain the designed data 

elements and the animated microstructure rendering software; 

FIG 20 shows a Web page incorporating an animated microstructure image. 

DETAILED DESCRIPTION OF THE INVENTION 

In principle, any of the methods described above for generating images incorporating informa- 
tion at the microstructure level can be used to create or display images with embedded ani- 
mated microstructures. However, since Artistic Screening is restricted to bi-color images 
(foreground color and background color) and since the method based on color differences 
requires considerable computing power, the preferred method for synthesizing images with 
embedded animated microstructures is based either on standard dithering or on the Multicolor 
Dithering method cited above. Hereinafter, the term dithering without the adjective "standard" 
or "multicolor" will refer to both standard dithering and Multicolor Dithering. Standard as well 
as Multicolor Dithering make use of a dither matrix, whose distribution of dither threshold val- 
ues represents the microstructure that will be part of the resulting target image (FIG 1A and 
FIG IB). Both standard dithering and Multicolor Dithering reproduce an input image (also 
called original or global image) in such a way that when seen from nearby, mainly the micro- 
structure embedded into the global image is visible, whereas when seen from far away, mainly 
the global image is visible (FIG 5). 

Standard dithering 

Multicolor Dithering is an extension of standard dithering. Standard dithering converts an 
intensity into a surface percentage. An intensity P(x) of foreground color C is compared with a 
dither threshold value G(x) and according to the comparison (see Fig 7A), if P(x) >G(x), the 
corresponding location x is set to the foreground color and if P(x) <=G(x), it is left as back- 
ground color. FIG 1A gives an example of a large dither matrix incorporating the microstruc- 
ture "GET READY"; FIG IB shows an enlarged part of it and FIG 2 represents the 
reproduction of uniform single color images at 20%, 40%, 60% and 80% foreground color 
intensity (the foreground color is represented as black). For more explanations on standard 
dithering, see H.R. Kang, Digital Color Halftoning, SPIE Press and IEEE Press, chapter 13, 
213-231. 

Multicolor dithering 

In Multicolor Dithering, a color C is rendered by a barycentric combination of several basic 
colors, for example the combination of 4 colors C a , C b , C c , and C d . Their respective relative 
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weights are d a , d b , d c , and d d (FIG 7B). Multicolor Dithering converts these relative weights 
into relative surface coverages. Multi-color dithering consists of determining the position of 
threshold value G in respect to intervals 0..d a , d a ..(d a + d b ), (d a + d b )..(d a + d b + d c ), (d a + d b + 
d c )..l, (see FIG 7C). According to the interval within which G is located, the dithered target 
image color C(x,y) will take value C a , C 5 , C c , or C d (see FIG 7C, color values along the x- 
axis). More precisely, if 0<=G<d a , C(x,y) = C a ; if d a <=G<(d a + d 5 ), C(x,y) = C b ; if (d a + 
d b )<=G<(d a + d b + d c ), C(x,y) = C c ; and if (d a + d b + d c )<=G<= 1, C(x,y) = C d . Best results are 
obtained by ordering the 4 basic colors C a , C b , C c , and C d located at the vertices of a tetrahe- 
dron according to their increasing CIE-LAB lightness values L*. 

The method for generating and displaying images incorporating animated microstructures 
requires the definition of the following elements (see FIG 8): 

- an original image (also called global image); 

- an original microstructure, preferably embodied as a dither matrix; 

- color information necessary for rendering the target animated microstructure image 
(optional); 

- a time-dependent animation transformation T t specifying an animated microstructure evolv- 
ing as a function of time; 

- a warping transformation T w specifying a warping between the animated or initial micro- 
structure and the warped microstructure (optional); 

and optionally a mask specifying the global image portions which are to be rendered with ani- 
mated microstructures as well as a possible blending between original image and pure ani- 
mated microstructure image, the blending allowing to specify microstructure appearance 
properties such as visibility, position and spatial extension of the microstructure. 

The original image is located in an original image space (x',y'), the original microstructure is 
located in an original microstructure space (also called original dither matrix space) (x",y"), 
the animated microstructure is located in an animated microstructure space (also called ani- 
mated dither matrix space) (u\v'), and the target animated microstructure image is located in 
the target animated microstructure image space, also simply called target image space (x,y). 

Herinafter, original image (x\y') may stand for original image space (x',y'), original micro- 
structure (x",y") may stand for original microstructure space (x",y"), animated microstruc- 
ture may stand for animated microstructure space (u\v') and target image (x,y) may stand for 
target image space (x,y). 

The microstructure may represent a text, a logo, a symbol, an ornament or any other kind of 
visual motive. Furthermore, the microstructure may combine several items, e.g. several sym- 
bols either identical or different, or a freely chosen combination of text, logos, symbols and 
ornaments. In the preferred cases of standard dithering and Multicolor Dithering, the micro- 
structure is defined by a dither matrix whose succession of dither threshold levels represent the 
desired visual motives (FIG IB). 

The time-dependent animation transformation T t may either be a time-dependent geometric 
transformation (e.g. translation, rotation, scaling, linear transformation, non-linear geometric 



6 




transformation) or any other time dependent transformation creating from at least one micro- 
structure a transformed microstructure whose shape varies in time. For example the time- 
dependent animation transformation may comprise a blending between two microstructure 
shapes (a shape blending method is described in M. Shapira, A. Rappoport, "Shape blending 
using the star-skeleton representation", IEEE Computer Graphics and Applications , Volume: 
15 Issue: 2 , March 1995, pp. 44 -50). In a preferred embodiment, the animation transformation 
T t provides the mapping between animated dither matrix space (u,v) and original dither matrix 
space (see FIG 12). 

The warping transformation T w (x,y) which provides a warping between the target image space 
(x,y) and the animated dither matrix space (u,v) may either be given by a formula allowing to 
obtain from a location (x,y) in the target image space the corresponding location (u,v) in the 
animated dither matrix space or by a program function returning for a given (x,y) coordinate in 
the final target image space the corresponding location (u,v) in the animated dither matrix 
space (see FIG 12, transformation T w (x,y)). Alternately, the warping transformation may be 
specified piecewise, by allowing the designer to specifiy a rectangular grid of control points 
and by allowing him to warp this grid as shown in FIG 13 A. 

The color information necessary for rendering the target animated microstructure image may 
comprise either an indication of which original image color layers {Cj} are to used for render- 
ing the target animated microstructure image or the specification of a set of basic colors {Cj} 
comprising possibly colors different from red, green and blue, cyan, magenta, yellow, white 
and black, with which the target image is to be synthesized. Colors members of the set of 
colors {Cj} used for animated microstructure image rendering are called hereinafter "basic 
colors". 

In the case of a mask with more than two levels of intensity, the mask's values specify a blending 
between the image rendered with microstructures, for example by dithering (standard or multi- 
color) and the color obtained by simple resampling of the original image according to the 
target's image size and resolution. Such a blending between the resampled original image and 
the image rendered with animated microstructures allows to produce less pronounced animated 
microstructures. It is also possible to apply a spatial blending between the scaled original image 
and the dithered image as described in the section "Image Rendering". In addition, the mask's 
values may evolve over time, yielding apparent changes in embedded microstructure 
appearance properties such as visibility, position or spatial extension. 

The method for generating an animated microstructure target image is formulated in general 
terms so as to encompass all methods capable of generating information at the microstructure 
level. However, in a preferred embodiment, either standard dithering or multicolor dithering is 
used. 

The method for generating a target image with an embedded animated microstructure com- 
prises the following steps (see FIG 8): 

(a) definition of elements required for generating the target image, i.e. an original image, an 
original microstructure (in a preferred embodiment, an original dither matrix), possibly color 
information specifying a set of basic colors {Cj} used for rendering the target animated micro- 
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structure image, a time-dependent animation transformation, possibly a warping transforma- 
tion and a mask; 

(b) traversing the target image (x,y) pixel by pixel and row by row, determining corresponding 
positions in the original image (x',y'), in the animated microstructure (preferred embodiment: 
animated dither matrix) (u,v), in the original microstructure (preferred embodiment: original 
dither matrix) (x",y") and in the mask; 

(c) obtaining from the original image position (x',y') the color C r to be reproduced, from the 
original microstructure (preferred embodiment: original dither matrix) space position (x",y") 
the rendering information (preferred embodiment: the dither threshold value G) and from the 
current mask position the corresponding mask value p; 

(d) carrying out the target image rendering algorithm (preferred embodiment: standard dither- 
ing or multicolor dithering) and determining output color C, possibly from the set of basic 
colors {Q}; 

(e) according to the mask value p, performing a blending between rendered (preferred embodi- 
ment: dithered) output color C and original image color C r 

If the mask value p indicates that the present image location does not need to be rendered with 
animated microstructures, then step (c) is modified to directly put color C r to be reproduced in 
the target image and steps (d) and (e) are skipped. If the mask is inexistant, then the whole 
image is reproduced with animated microstructures. 

The original image may be a simple RGB color image stored in any known format. The micro- 
structure, in a preferred embodiment, the dither matrix, is either precomputed and ready to use 
or has been created applying one or several tools. A known technique of converting bi-level 
microstructures such as typographic characters, logos or ornaments into dither matrices com- 
prises (a) low-pass filtering in order to create a multi-level image, (b) renumbering of the 
image intensity levels in order to create the dither levels of the dither matrix, and (c) matrix 
equilibration in order to improve the dither matrix quality (see V. Ostromoukhov, R.D. Hersch, 
"Multi-Color and Artistic Dithering", Siggraph'99, Computer Graphics Proceedings, Annual 
Conference Series, 1999, pp. 425-432). 

Generation of animated microstructure images by standard dithering 

It is however possible to generate images with animated microstructures by applying the stand- 
ard dithering method independently to one or several basic colors. A basic color may be the set 
of red, green and blue or any other set of colors by which the image is described (for example 
cyan, magenta, yellow and black). One may apply standard dithering to one, several or all 
basic colors. As an example, one may apply standard dithering separately to the red, green and 
blue layers of an image (FIG 14A and FIG 14B) and display the resulting target image by 
superposing the dithered red, green and blue layers. The resulting target image will thus be ren- 
dered with red, green, blue, cyan (overlap of green and blue), magenta (overlap of red and 
blue), yellow (overlap of red and green) and white (overlap of red, green and blue), see FIG 
14C. Instead of applying standard dithering to red, green and blue as in the previous example, 
one may also apply standard dithering to one of the color layers, for example the predominant 
color layer or the color layer dominant in the image part where one would like to insert the 
microstructure. For example, in order to insert a microstructure in the sky, one may choose to 
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apply standard dithering to the blue layer (FIG 15B) and leave the other color layers 
unchanged, i.e. generate them by simple resampling of the original image (FIG 15A) accord- 
ing to the target image size and resolution (FIG 15C). In that case, target image pixels are com- 
posed of a blue color layer obtained with the standard dithering method and red and green 
layers obtained by resampling the original image. 

Generation of animated microstructure images by Multicolor Dithering 

In the preferred embodiment of generating animated microstructure images by Multicolor 
Dithering, the method comprises initialization steps, rendering steps and an image display step. 

The initialization steps comprise (a) the creation of a selection mask M(x,y) associated with 
the target image, deduced from the original mask M'(x\y') associated with the original image, 
(b) intialization for the color separation of the original image (e.g. RGB) according to the 
selected set of basic colors, (c) the creation of a data structure facilitating the color separation, 
(d) carrying out the color separation and associating in a color separation map to each target 
color image pixel the basic colors with which it is to be color dithered and their associated 
basic colors weights, (e) associating in a warping transform map to each location (x,y) within 
the target image space a pointer to the corresponding location in the animated dither matrix 
space according to the user defined warping transformation. Steps (c), (d) and (e) are useful for 
speeding up image rendition. As a variant, one may choose to carry out the color separation 
and possibly the warping transform during image rendering. 

The target image selection mask M(x,y) is obtained by traversing the target image space (x,y) 
pixel by pixel, finding corresponding locations (x',y') in the original image space, and interpo- 
lating corresponding mask values from neighbouring original image pixel mask values 
M'(x',y'). Nearest neighbour interpolation associates one original image pixel mask value M' 
with each target image pixel mask value M. Bi-linear interpolation allows to interpolate 
between original image pixel mask values. 

Several methods for carrying out the color separation exist: one may solve the Neugebauer 
equations for the set of outpout colors (see for example H.R. Kang, Color Technology for Elec- 
tronic Imaging Devices, SPIE Optical Engineering Press, 1997, Chapter 2, Section 1, pp. 34- 
40) or place the output colors in an output color space, e.g. CIE-XYZ and tetrahedrize that 
space (see S.M. Chosson, R.D. Hersch, Visually-based color space tetrahedrizations for print- 
ing with custom inks, Proc. SPIE, 2001, Vol.4300, 81-92). In that case, the preferred data struc- 
ture facilitating the color separation is a 3D grid data structure pointing to the tetrahedra 
intersecting individual grid elements. 

In the case that the selected basic colors are located in a rectilinear grid, the tetrahedrization is 
straightforward : each cube or rectlinear volume element comprising 8 vertices can be decom- 
posed into 6 tetraheders (see H.R. Kang, Color Technology for Electronic Imaging Devices, 
SPIE Optical Engineering Press, 1997, Section 4.4 Tetrahedral interpolation, pp 70-72). If the 
designer is allowed to choose any set of basic colors, the tetrahedrization is slightly more com- 
plex, but can be carried out without difficulty with prior art methods (see for example the book 
Scientific Visualization : Overviews, Methodologies, and Techniques, by Gregory M. Nielson, 
Hans Hagen, Heinrich Muller, Mueller (eds), IEEE Press, Chapter 20, Tools for Triangulations 
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and Tetrahedrizations and Constructing Functions Defined over Them, pp. 429-509). 

In the case that the color separation is carried out by tetrahedrization, each target image pixel 
color is rendered by 4 basic colors, members of the selected set of the basic colors. For com- 
puting the 4 basic colors associated with each target image pixel (x,y), the color C r at the corre- 
sponding original image location (x',y') is determined by resampling, i.e. by interpolating 
between colors of neighbouring original image pixels (e.g. prior art nearest neighbour or bi-lin- 
ear interpolation). Resampled color C r is used to find the tetrahedron which encloses it. The 4 
basic colors C a , C b , C c , C d located at the tetrahedron's vertices and their barycentric weights 
d a , d b , d c , d d allowing to render resampled original image color C r according to 

C r = d a C a + d b C b + d c C c + d d C d should be stored, possibly together with original image resam- 
pled color C r , in a target image color separation map. 

The image rendering steps are as follows. 

For rendering successive target image instances of the target animated microstructure image, 
for each target image instance, we traverse the target image space pixel by pixel by traversing 
one pixel row after the other. For each target pixel (x,y), if the target image mask value M(x,y) 
indicates that multi-color dithering is to be applied, (e.g. M(x,y)<>0), we read from the target 
image color separation map the basic colors and their respective weights. We determine the 
dither threshold value G associated with a target pixel (x,y) by obtaining the pointer to the cor- 
responding location (u,v) in the animated dither matrix space, for example by accessing the 
warping transform map created in the initialization phase and from there, by applying the cur- 
rently valid transformation T t (u,v), we obtain the current location (x",y") within the original 
dither matrix space. The threshold value G(x",y"), the basic colors C a , C b , C c , C d and their 
respective weights d a , d b , d c , d d are used for multicolor dithering. Multi-color dithering con- 
sists of determining the position of threshold value G with respect to intervals 0..d a , d a ..(d a + 
db)> (d a + d b )..(d a + d b + d c ), (d a + d b + d c )..l. According to the interval within which G is located, 
the dithered target image color C(x,y) will take value C a , C b , C c , or C d (see FIG 7C and sec- 
tion "Multicolor dithering" above). In the case that standard dithering is used instead of multi- 
color dithering, we determine as above the dither threshold value G and use it to compare it 
with the intensity of the basic color (or colors) to be dithered and according to the comparison 
(see section "Standard dithering" above), use that basic color (or colors) to render the current 
target image pixel (x,y). FIG 15B and FIG 15C show how dithering can be applied to one of 
the image's color's, namely blue. 

For each new target image instance to be rendered, the animation transformation T t (x,y) 
describing the mapping between the animated dither matrix space and the original dither 
matrix space is updated. 

In the case of a mask M(x,y) specifying discrete values representing a proportion p between 0 
and 1, the final color Cf (x,y) is a combination of the dithered color C(x,y) and of the original 
resampled color C r , for example C f (x,y) = p C(x,y) + (1-p) C r . Instead of a pixel-wise blend- 
ing between dithered image color C(x,y) and original resampled color C r , it is also possible to 
apply a spatial blending, i.e. to ensure that only proportion p of neighbouring pixels take the 
dithered color C(x,y) and proportion (1-p) takes the original resampled color values C r . For 
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this purpose, one can use for example a spatial dispersed dither matrix D(x,y), e.g. Bayer's 4x4 
dither matrix (FIG 16A) and use thresholds t=0,l,2..15 to decide if a pixel should take the 
original resampled color value C r , when p =< t/16 or take the dithered color C when p > t/16. 
As an illustration of spatial blending, FIG. 16B shows in one-dimensional space the compari- 
son between the proportion p(x) and the dither values D(x): where p(x) > D(x), the correspond- 
ing segment (black in FIG 16B) takes the dithered image color values C(x) 
and where p(x) <=D(x), the corresponding segment (white in FIG 16B) takes the original resa- 
mpled color values C r (x). 

The display step comprises the display of successively generated animated microstructure 
image instances. 

A schematic example 

As an example let us assume FIG 9A represents the original color image. FIG 9B represents 
the dither matrix paving the original dither matrix space. The time or frame-dependent anima- 
tion transformation T t maps the animated dither matrix within an animated dither matrix space 
into the original dither matrix space. FIG 10A represents a warping grid laid out over the ani- 
mated dither matrix space. In FIG 10B, the warped grid is shown in the target image space. 
The warping transformation T w allows to map locations from the target image space into corre- 
sponding locations in the animated dither matrix space. FIG. 11 A shows a mask specifiying 
which part of the original image needs to be rendered by animated microstructures. FIG 11B 
shows schematically the rendered target color image space, where the part covered by the mask 
is rendered with animated microstructures. If the animation transformation is a simple vertical 
translation as a function of time, the microstructure will move vertically in successive dis- 
played frames. The "LSP" microstructure is obtained thanks to the warping transformation 
(FIG 13A) which transforms for example the repetitive microstructure shown in FIG 13B into 
the warped microstructure shown in FIG 13C. 



A real example 

As real example, FIG 1. shows a dither matrix comprising the "GET READY" microstructure 
shapes. FIG 2. shows the microstructure obtained by dithering with constant foreground color 
intensity levels of 20%, 40%, 60% and 80% (the foreground color is shown in black, the back- 
ground is represented by the paper white). FIG 3. shows the original image, with a superim- 
posed warping grid (the grid is made of rectangular elements, with one additional diagonal per 
rectangle defining two triangles; the triangles are used for the warping transformation). In the 
present case, the warping grid has the effect of shrinking the microstructure at the bottom and 
top of the image. FIG. 4 shows the bi-level mask specifying the regions to be rendered with a 
animated microstructure and FIG 5 shows one instance of the resulting image comprising an 
animated microstructure in the regions specified by the mask. One can easily perceive the 
microstructure made of the warped "GET READY" shapes. FIG 6 shows several instances of 
the rendered animated microstructure image, i.e. the rendered animated microstructure image 
at different time points. The animation transformation moves the microstructure up and down 
and at the same time displaces it slowly to the left. The animation transform T t of this example 
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has the form 

jc" = s x (u + k u - i) 

(a (, . x360Yi 

y" = s y yy + A * COSTS' • I + J J 

where i is the number of the current target image instance, s is the wave oscillating speed, k u is 
the horizontal translation speed, A is the horizontal period of the microstructure wave, A is its 
amplitude and s x , s y represent respectively horizontal and vertical scaling factors. The cosinu- 
soidal vertical displacement of the microstructure depends on its current location w, i.e. there is 
a phase difference in the vertical displacement of the microstructure at different horizontal 
locations. 



Generalization to image animations (moving images, videos) 



The method described above can be easily generalized to moving images, image animations 
and videos made of individual frames. Each frame will incorporate a visible microstructure. 
Frames with animated microstructure are created by rendering successive frames as separate 
images with an animated microstructure. Instead of a single original image, successive frames 
are used as original images and rendered using the animated microstructure defined by the 
dither matrix, the animation transformation, the warping transformation, the set of basic colors 
{Cj} and a possible mask layer comprising one or several masks, possibly up to one mask per 
frame. The mask values evolving over time yield apparent changes in embedded microstruc- 
ture appearance properties such as visibility, position or spatial extension. These mask values 
may be adapted according to moving visual elements of the image animation. For example 
when the moving camera recording the image animation follows a river, the evolving mask 
shape may take the evolving shape of the river, allowing to confine the evolving microstructure 
within the moving river present in the image animation. 



Computing system for displaying images with animated microstructure 



Images with animated microstructures can be synthesized offline by a computer running an 
animated microstructure image rendering software. The resulting image animation can then be 
incorporated into Web pages as animated images (e.g. animated GIF or MNG formats). An 
alternative consists in creating an image computing and display system, for example an applet, 
running the animated microstructure image rendering software. In that case, the image comput- 
ing and display system will run on the client's computer and display the animated microstruc- 
ture image or image animation. As a preferred embodiment, the image computing and display 
system will receive from the server computing system (FIG 17) as input data the input color 
image, the dither matrix, the animation transformation, the warping transformation, the set of 
basic colors {Cj} and a possible mask layer. With the present technology, the preferred embod- 
iment of an image computing and display system is a Java applet. The image computing and 
display system's program (e.g. the program running as an applet) will then generate and dis- 
play the target image by carrying out the initialization, image rendering and image display 
steps described above. 



In addition, specific embodiments of the animated microstructure image rendering system may 
allow to tune some of the image rendering parameters according to user preferences or user 
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profiles. For example one image selected from a set of images, one set of basic colors selected 
from various sets of basic colors, one dither matrix selected from different dither matrices, one 
animation transformation and possibly a warping transformation may be tuned according to 
user preferences or profiles. These specific embodiments allow to customize the animated 
microstructure images according to users or user categories. 



Server system for designing images with animated microstructure 

Optionally, a specific server (e.g. a Web site) can be conceived which allows designers to cre- 
ate images with animated microstructures on their own computers (FIG. 18). The program 
interface running on their computers (e.g. dynamic Web page incorporating an applet) will 
exchange information with the server. With such a Web based design interface, designers or 
graphists may specifiy or create the source image, the dither matrix, the basic colors, the ani- 
mation transform, the warping transform and the image mask layer. By being able to modify 
interactively each of these parameters and elements, and immediately visualizing the results, 
designers may be able to interactively create appealing images with animated microstructures. 
Upon signing a licensing agreement, they may then receive the authorization to transfer the 
animated microstructure rendering software (e.g. the applet's code) as well as the created data 
elements into their own Web pages (FIG 19). FIG. 20 shows an image with animated micro- 
structure incorporated into a Web page. 

Software for offline design of animated microstructure images 

Alternately, clients interested in creating animated microstructure images may sign a licensing 
agreement and receive the software allowing them to locally conceive the elements necessary 
for creating images with animated microstructures as described above. These elements may 
then be sent to the Web server which upon signing a licensing agreement, may then issue an 
applet able to run the so-created animated microstructure image. 
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